Papers with retrieval model
INDUS: Effective and Efficient Language Models for Scientific Applications (2024.emnlp-industry)
Copied to clipboard
Bishwaranjan Bhattacharjee, Aashka Trivedi, Masayasu Muraoka, Muthukumaran Ramasubramanian, Takuma Udagawa, Iksha Gurung, Nishan Pantha, Rong Zhang, Bharath Dandala, Rahul Ramachandran, Manil Maskey, Kaylin Bugbee, Michael Little, Elizabeth Fancher, Irina Gerasimov, Armin Mehrabian, Lauren Sanders, Sylvain Costes, Sergi Blanco-Cuaresma, Kelly Lockhart, Thomas Allen, Felix Grezes, Megan Ansdell, Alberto Accomazzi, Yousef El-Kurdi, Davis Wertheimer, Birgit Pfitzmann, Cesar Berrospi Ramis, Michele Dolfi, Rafael Lima, Panagiotis Vagenas, S. Mukkavilli, Peter Staar, Sanaz Vahidinia, Ryan McGranaghan, Tsengdar Lee
| Challenge: | Large language models trained on general domain corpora showed remarkable results on natural language processing tasks. |
| Approach: | They develop a suite of large language models trained on general domain corpora that address NLP tasks and smaller versions of them created using knowledge distillation. |
| Outcome: | The proposed models outperform general-purpose and domain-specific encoders on new and existing tasks and in industrial settings. |
Plot Retrieval as an Assessment of Abstract Semantic Association (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing information retrieval datasets cannot capture abstract semantic associations well. |
| Approach: | They propose a task that retrieves relevant plots from the book for a query using a labeled dataset. |
| Outcome: | The proposed task can be used to evaluate the performance of IR models on the novel task Plot Retrieval. |
Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval (2022.tacl-1)
Copied to clipboard
| Challenge: | Current approaches to cross-modal retrieval process text and visual input jointly . current approaches are pretrained from scratch and suffer from huge retrieval latency and inefficiency issues . |
| Approach: | They propose a cooperative retrieve-and-rerank framework that turns pretrained text-image multi-modal models into efficient retrieval models. |
| Outcome: | The proposed framework improves retrieval performance over current approaches . it uses twin networks to encode all items of a corpus and a cross-encoder component for a more nuanced ranking . |
Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation (2021.acl-short)
Copied to clipboard
| Challenge: | Early fusion models with cross-attention have shown better-than-human performance on some question answer benchmarks, while it is a poor fit for retrieval since it prevents pre-computation of the answer representations. |
| Approach: | They propose a supervised data mining method to train an efficient late fusion retrieval model by using cross-attention models with cross-references. |
| Outcome: | The proposed model outperforms retrieval models trained with gold annotations on Precision at N (P@N) and Mean Reciprocal Rank (MRR). |
Interactive Text-to-Image Retrieval with Large Language Models: A Plug-and-Play Approach (2024.acl-long)
Copied to clipboard
| Challenge: | primarily addressed in text-to-image retrieval task using dialogue-form context query . conventionally, text-based retrieval methods rely on initial text descriptions . |
| Approach: | They propose a plug-based retrieval method that uses large language models as questioners to generate non-redundant questions about the attributes of the target image. |
| Outcome: | The proposed method performs better than zero-shot and fine-tuned baselines in benchmarks. |
Ensuring Safe and High-Quality Outputs: A Guideline Library Approach for Language Models (2024.naacl-long)
Copied to clipboard
Yi Luo, Zhenghao Lin, YuHao Zhang, Jiashuo Sun, Chen Lin, Chengjin Xu, Xiangdong Su, Yelong Shen, Jian Guo, Yeyun Gong
| Challenge: | Guide-Align is a guideline-oriented approach to augment the safety and quality of Large Language Models. |
| Approach: | They propose a guideline-oriented method to augment the safety and quality of large language models. |
| Outcome: | The proposed method outperforms existing methods on three benchmarks and shows significant improvements in security and quality. |
Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing (P19-1)
Copied to clipboard
| Challenge: | a context-aware retrieval model and a meta-learning paradigm are used for context-dependent semantic parsing . |
| Approach: | They propose a retrieval model and a meta-learner to incorporate retrieved datapoints as context-dependent semantic parsing evidence. |
| Outcome: | The proposed approach performs better than retrieve-and-edit baselines on CONCODE and CSQA datasets. |
Chain-of-Skills: A Configurable Model for Open-Domain Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Using customized retrieval models, model transferability and scalability are limited. |
| Approach: | They propose a modular retrieval model where individual modules correspond to key skills that can be reused across datasets. |
| Outcome: | The proposed model outperforms self-supervised retrievers in zero-shot evaluations and achieves state-of-the-art fine-tuned retrieval performance on NQ, HotpotQA and OTT-QA. |
A Dataset and Baselines for Multilingual Reply Suggestion (2021.acl-long)
Copied to clipboard
| Challenge: | Reply suggestion models help users process emails and chats faster. |
| Approach: | They present a multilingual reply suggestion dataset with ten languages . they build a generation model and a retrieval model as baselines for MRS . |
| Outcome: | The proposed model complements existing benchmarks for cross-lingual generalization . the model has different strengths in the English monolingual setting and requires different strategies to generalize across languages. |
DPTDR: Deep Prompt Tuning for Dense Passage Retrieval (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies show that prompt tuning is unfriendly for industrial deployment in dense retrieval tasks. |
| Approach: | They propose to apply prompt tuning to dense retrieval tasks to reduce deployment cost . they propose to use retrieval-oriented intermediate pretraining and unified negative mining . |
| Outcome: | The proposed method outperforms state-of-the-art models on MS-MARCO and Natural Questions. |
Constructing Multi-Modal Dialogue Dataset by Replacing Text with Semantically Relevant Images (2021.acl-short)
Copied to clipboard
| Challenge: | Existing training methods for multi-modal dialogue systems rely on image captioning or visual question answering datasets that are irrelevant to the dialogue context. |
| Approach: | They propose to create a 45k multi-modal dialogue dataset with minimal human intervention . they use text dialogue datasets, image-mixed dialogues and contextual-similarity filtering . |
| Outcome: | The proposed dataset can be used as training data for multi-modal dialogue systems . human evaluations show that the model can be effectively used . |
Retrieving Relevant Context to Align Representations for Cross-lingual Event Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual transfer learning for event detection are mixed with event-discriminative context. |
| Approach: | They propose a method where representations are augmented with additional context to bridge the gap between languages while enriching contextual information to facilitate ED. |
| Outcome: | The proposed model performs well on three languages. |
“None of the Above”: Measure Uncertainty in Dialog Response Retrieval (2020.acl-main)
Copied to clipboard
| Challenge: | End-to-end (E2E) dialog retrieval models jointly encode a dialog and a candidate response, assuming the ground truth is always present in the candidate set. |
| Approach: | They propose to capture the original retrieval model's confidence concerning the best prediction using trivial additional computation. |
| Outcome: | The proposed model can capture the model's confidence concerning the best prediction using trivial additional computation. |
Nearest Neighbor Zero-Shot Inference (2022.emnlp-main)
Copied to clipboard
| Challenge: | Using non-parametric memory for retrieval-augmented language models yields significant performance boosts over strong zeroshot baselines. |
| Approach: | They propose a retrieval-augmented language model with fuzzy verbalizers that expands the verbalizes that define different end-task class labels. |
| Outcome: | The proposed model outperforms non-retrieval-augmented language models on perplexity-based evaluations but gains transfer marginally . the main challenge is to achieve coverage of the verbalizer tokens that define the different end-task class labels. |
LegalSearchLM: Rethinking Legal Case Retrieval as Legal Elements Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on legal case retrieval have limited results . limited representations and legally irrelevant matches are often used . |
| Approach: | They propose a large-scale Korean LCR benchmark and a retrieval model that performs legal element reasoning over the query case. |
| Outcome: | a new model outperforms baseline models on a Korean LCR benchmark . it performs state-of-the-art on 411 diverse crime types in queries over 1.2M candidate cases . previous studies have shown that the model can generalize to out-of domain cases if it is trained on in-domain data . |
Explanations for CommonsenseQA: New Dataset and Models (2021.acl-long)
Copied to clipboard
Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, Dinesh Garg
| Challenge: | a dataset called CommonsenseQA (CQA) was recently released to advance the research on common-sense question answering (QA) |
| Approach: | They propose to retrieve and generate explanations for a given question, correct answer choice, incorrect answer choices tuple from a dataset called CommonsenseQA. |
| Outcome: | The proposed model beats baseline model by 100% in F1 score and similarity score of 61.9 . |
Multi-stage Retrieve and Re-rank Model for Automatic Medical Coding Recommendation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for ICD indexing have a heavy label distribution and a manual process . Xie and Xing (2017) propose a new approach to ICD re-ranking . |
| Approach: | They propose a "retrieve and re-rank" framework to allocate subsets of ICD codes to medical records . they leverage auxiliary knowledge of the electronic health records (EHR) and a discrete retrieval method . |
| Outcome: | The proposed method achieves state-of-the-art performance on the MIMIC-III benchmark. |
Distilling the Knowledge of Large-scale Generative Models into Retrieval Models for Efficient Open-domain Conversation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | generative models are less practical for building real-time conversation systems due to high latency and large memory footprint. |
| Approach: | They propose a method that preserves the efficiency of a retrieval model while leveraging the conversational ability of generative models. |
| Outcome: | The proposed method preserves the efficiency of a retrieval model while leveraging the conversational ability of generative models. |
CoRT: Complementary Rankings from Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent approaches to information retrieval mitigate computational costs by using a multi-stage ranking pipeline. |
| Approach: | They propose a ranking model that leverages contextual representations from pre-trained language models to complement term-based ranking functions while causing no significant delay at query time. |
| Outcome: | The proposed model significantly increases candidate recall by complementing BM25 with missing candidates while causing no significant delay at query time. |
TOME: A Two-stage Approach for Model-based Retrieval (2023.acl-long)
Copied to clipboard
| Challenge: | Recent research has focused on model-based retrieval, which discards the index in the traditional retrieval model and memorizes the candidate corpora using model parameters. |
| Approach: | They propose a model-based retrieval approach that discards the index in the traditional retrieval model and memorizes the candidate corpora using model parameters. |
| Outcome: | The proposed approach eliminates the index in the traditional retrieval model and memorizes the candidate corpora using model parameters. |
VizoMem: A Visual-Textual Memory Framework for Efficient Long-Horizon Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing systems that use long-context modeling incur computational and memory overhead. |
| Approach: | They propose a visual memory framework that pre-rendered text into structured images and stored as visual notes for agentic systems. |
| Outcome: | The proposed system reduces token consumption while preserving effective long-term memory recall. |
An Exploratory Study on Long Dialogue Summarization: What Works and What’s Next (2021.findings-emnlp)
Copied to clipboard
Yusen Zhang, Ansong Ni, Tao Yu, Rui Zhang, Chenguang Zhu, Budhaditya Deb, Asli Celikyilmaz, Ahmed Hassan Awadallah, Dragomir Radev
| Challenge: | Existing models for dialogue summarization focus on extracting the main events of short conversations, but real-world dialogues are difficult to train. |
| Approach: | They propose three strategies to deal with the lengthy input problem and locate relevant information using long dialogue datasets. |
| Outcome: | The retrieve-then-summarize pipeline models yield the best performance on three long dialogue datasets. |
Meet Your Favorite Character: Open-domain Chatbot Mimicking Fictional Characters with only a Few Utterances (2022.naacl-main)
Copied to clipboard
| Challenge: | In this paper, we consider mimicking fictional characters as a promising direction for building engaging conversation models. |
| Approach: | They propose a task where only a few utterances of each fictional character are available to generate responses mimicking them. |
| Outcome: | The proposed method generates responses better reflecting the style of fictional characters than baseline methods. |
REPLUG: Retrieval-Augmented Black-Box Language Models (2024.naacl-long)
Copied to clipboard
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih
| Challenge: | Existing retrieval-augmented language models require access to internal representations to enhance performance. |
| Approach: | They introduce a retrieval-augmented language modeling framework that treats the language model as a black box and augments it with a tuneable retrieval model. |
| Outcome: | The proposed framework improves performance on language modeling tasks by 6.3% and 5.1%. |
Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Question Answering models typically use retrieval and reasoning components to identify relevant information for reasoning. |
| Approach: | They propose a retrieval parameterization method that marginalizes unanswerable queries . they show that marginalization allows a model to mitigate false negatives in annotations . |
| Outcome: | The proposed model improves on two multi-document question answering datasets and shows that marginalization improves performance. |
SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation (2025.emnlp-main)
Copied to clipboard
Qian Dong, Jia Chen, Qingyao Ai, Hongning Wang, Haitao Li, null Yiwu, Yao Hu, Yiqun Liu, Shaoping Ma
| Challenge: | Existing retrieval-augmented code generation methods fail to accurately fetch the knowledge required for code generation for consecutive code fragments. |
| Approach: | They propose a paradigm that enables large language models to Self-express their information needs to enhance retrieval-augmented code generation methods. |
| Outcome: | Experiments show that SelfRACG can retrieve external knowledge that better aligns with the LLM’s own information needs, resulting in superior generation performance compared to vanilla RACG. |
CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that dense retrieval models are better at obtaining query-informed representations. |
| Approach: | They propose a dual-encoder approach that computes latent representations of query and document independently, but inference replaces the real query with a generated one. |
| Outcome: | The proposed approach outperforms previous dense retrieval models on in-domain and out-of-domain datasets. |
DMRetriever: A Family of Models for Improved Text Retrieval in Disaster Management (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models fail to handle the varied search intents inherent to disaster management scenarios, resulting in inconsistent and unreliable performance. |
| Approach: | They propose a new series of dense retrieval models tailored for disaster management that train on a three-stage framework with unsupervised contrastive pre-training and difficulty-aware progressive instruction fine-tuning. |
| Outcome: | The proposed model outperforms baseline models by 13.3 times and 33 times over baselines with only 7.6% of their parameters. |
Dense Passage Retrieval: Is it Retrieving? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) internally store repositories of knowledge, but access to these repositoriels is imprecise. |
| Approach: | They propose a paradigm called retrieval augmented generation to address hallucinations . they analyze the role of fine-tuning pre-trained networks to enhance alignment . |
| Outcome: | The proposed paradigm addresses hallucinations by fine-tuning pre-trained models . the model can be decentralized, inject facts as decentralized representations . |
A Counterfactual Explanation Framework for Retrieval Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing literature on explainability of information retrieval has focused on illustrating the concept of relevance concerning a retrieval model. |
| Approach: | They propose to add terms to a document to improve its ranking to answer the question of which words played a role in not being favored by a retrieval model. |
| Outcome: | The proposed framework predicts counterfactuals for statistical and deep-learning models. |